{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# COMPSCI 389: Homework 4\n", "\n", "**Assigned**: April 9, 2024. **Due**: April 15, 2024 at 2:00pm Eastern. **Note**: Submissions received after 2:00pm Eastern on April 24, 2024 will receive no credit.\n", "\n", "**Submitting**: Upload your submission on Gradescope as a `.pdf`. Converting to a PDF can be a complicated process, and so we encourage you to test this process well in advance of the submission deadlines. We recommend converting to HTML, opening the HTML file in a browser, and then printing or exporting to a PDF from your browser. We do not recommend directly converting to a PDF, since this requires installing xelatex. To convert to HTML in VSCode, press `ctrl+shift+p` and type `export`, and you should see an option to export to HTML.\n", "\n", "**Note**: Keep your `.ipynb` file, as we may request it directly (via email).\n", "\n", "**Note**: When converting to a PDF file, ensure that all of your code cells have been executed. The results of these executions *must* be included in your submitted PDF." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## NEW Instructions\n", "\n", "\n", "This assignment contains 11 questions, each worth 15 points. However, your score on this assignment is capped at 100 points. You can receive full credit by answering just seven questions correctly, although we recommend answering all questions (and all questions may appear on the test).\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Instructions\n", "\n", "Complete the questions below, replacing the blue text with your own answers (your answers do not need to remain in blue). Do **not** modify the green text. Try to answer the questions without consulting your notes or any online material. If you cannot, then consult your notes, and if absolutely necessary, consult course materials (slides, notebooks) and/or Wikipedia. Do **not** use other sources or tools like ChatGPT. Complete this part of the assignment on your own (do **not** work with others).\n", "\n", "After you have completed all of the questions, at the bottom of this assignment you will find a link to another notebook, `Homework 4 Solutions.ipynb`. This contains the solutions, and instructions for ensuring that your answers are correct and sufficient. Make another pass through your homework assignment, replacing the green text with descriptions of what you missed for each question, and providing the fixes necessary to make your answer correct. **The solutions file may include additional instructions, which may include additional content to respond to even if you got a question correct (e.g., additional reflection).** During this second stage where you are filling in your answers, replacing the green text, you may reference the solutions, work with others, and use any tools (including ChatGPT).\n", "\n", "You will only submit this assignment once after replacing both the blue and green text. You do not need to submit the assignment between the first and second passes. Grading for each question will be based on whether you followed this process, and arrived at the correct answers and have sufficient discussion/text in the end. Points will be deducted if you did not make a reasonable effort to answer the question initially, if your final answer remains incorrect, of if your answers were not sufficiently clear (so, write in full sentences with proper punctuation, and conveying your arguments clearly). Other than verifying that you made a reasonable initial effort for your initial answers (blue), points will **not** be deducted due to *initial* answers being incorrect. Hence, there is no reason to break the rules to obtain correct answers initially." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Part 1: Multiple Choice and Short Answer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 1. [15 points] True or False: Overfitting occurs when a parametric model is trained so much that it achieves low error on the **testing** set, but will have high error when applied to new points.\n", "\n", "**Warning (to inform decisions about which problems to complete)**: This question requires additional reflection in the green answer.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer, which should be a single word: True or False.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2. [15 points] When using an artificial neural network for a classification problem with $m>2$ classes, how many outputs does the network typically have? Why?\n", "\n", "Note: The solution to this question is two sentences - we are not looking for a long answer.\n", "\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3. [15 points] Why do we often view classifiers as stochastic (producing a probability of each label given an input) rather than deterministic during training?\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 4. [15 points] For binary classification tasks, parametric models with a single output can be used. Explain how one output can be enough to select between two labels.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 5. [15 points] The Cross-Entropy Loss is $\\operatorname{Loss}(w,D)=-\\frac{1}{n}\\sum_{i=1}^n \\ln\\left ( \\Pr(Y_i = \\hat Y_i) \\right )$. Explain this loss in English: why does minimizing this loss result in models that make good predictions?\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 6. [15 points] Give an example where a parametric model with $99\\%$ accuracy corresponds to a model with *poor* performance.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 7. [15 points] You are tasked with making an ML system for predicting whether a tumor is benign or malignant. If the tumor is predicted to be benign, the patient will be sent away and checked again in another few years. If the tumor is malignant, a human doctor will look at the results and determine how to proceed. Propose a loss function for this task and explain why it would be appropriate.\n", "\n", "**Warning (to inform decisions about which problems to complete)**: This question requires significant additional reflection in the green answer.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 8. [15 points] Can the accuracy of a model be computed from the confusion matrix? If no, why not, and if yes, how?\n", "\n", "Note: For simplicity, assume the confusion matrix reports the number of samples falling into each category, not the probability.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 9. [15 points] What are the two main components of a VAE, and what does each do?\n", "\n", "Note: We are looking for one sentence per component.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 10. [15 points] What are the two main components of a GAN, and what does each to?\n", "\n", "Note: We are looking for one sentence per component.\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 11. [15 points] We described how VAEs include a term in the loss function that encourages the latent representation of the training data to be normally distributed in the latent space. What tends to go wrong when this term is omitted?\n", "\n", "***Initial Answer***\n", "\n", "Replace this text with your answer.\n", "\n", "---\n", "\n", "***Updated Answer***\n", "\n", "Replace this text with your response to the solution document.\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The solutions can be found here: [https://people.cs.umass.edu/~pthomas/courses/COMPSCI_389_Spring2024/Homework%204%20Solutions.ipynb](https://people.cs.umass.edu/~pthomas/courses/COMPSCI_389_Spring2024/Homework%204%20Solutions.ipynb)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.7" } }, "nbformat": 4, "nbformat_minor": 2 }